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The state of Georgia not only assesses the generic 
teaching competencies that are necessary for all teachers seeking 
certification, but also administers 28 separate criterion referenced 
tests, each for a different content-specialty area. Among the most 
recent of these tests is the Reading Specialist Test, the development 
of which involved ascertaining the domain of knowledge required of 
reading specialists, analyzing their job activities, and constructing 
an appropriate test. A study undertaken in the fall of 1982 examined 
the correspondence between the content objectives of the Georgia 
Reading Specialist Test and the content emphasized in graduate-level 
reading education courses in Georgia colleges and universities. A 
questionnaire was sent to all faculty teaching at least one graduate 
level reading course with a state-approved program in reading at the 
master's level or higher. The 118 job-related content objectives in 
the six subareas of the specialist test (language development and 
reading readiness, reading skills, developmental reading instruction, 
assessment, reading disabilities, and program management) were 
collapsed and reduced to 56 questionnaire items. Generally the 
results suggested that the reading specialist candidates would do 
well on most of the objectives. Subarea responses looked very 
positive for all subareas, with the possible exception of reading 
skills. (HOD) 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document* * 
*********************************************************************** 



ERIC 



U.S. MP ART Ml NT OF EDUCATION 

NATIONAL INSTITUTE OF 60UCATI0N 

EDUCATIONAL RESOURCES INFORMATION 

CENTER lERlCl 
v/lh« tlocuMHNii rui b*m (epiodtibed « 
/^»ec»-.wrtl Itom »»« p«i\o« ot o»ijjm«lK>n 

M*u>t cfui«i^ rwvf iifvn nu(i»« to «»»piov« 

n-tmxlucNx* UtiJUty 

• ?o*\\% ol wtew o» options *tate<l »n thit rioc»» 
n*nt ilonol fi*xe*«»tlv tcptcurni oM<v»»NlE 



Teacher Competency Testing and 

Reading Specialty Preparation in Georgia 

Rona F. Flippo 
Georgia Department of Education 

David A. Hayes 
University of Georgia 

Robert L. Aaron 
University of Georgia 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



Rona F. Flippo 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



Running head: Teacher Competency Testing 



Teacher Competency Testing - 1 



Teacher Competency Testing and 
Reading Specialty Preparation in Georgia 



In an effort to control the quality of instruction offered by public 
schools, state departments of education across the nation are developing 
ways to test the competency of individuals who seek certification in their 
teaching specialties. This competency testing exists at various stages 
of development and at various levels of sophistication in many states. 
For example, Georgia, Oklahoma, Alabama, South Carolina, Florida, and 
Arizona have existing teacher competency test programs, while West Virginia, 
California, Maryland, Connecticut, Kentucky, New York, and Pennsylvania 
are in the process of developing programs or plans for programs. Georgia, 
Oklahoma, and Alabama have developed separate criterion-referenced tests 
for most of the areas in which they issue certificates. South Carolina 
has developed criterion-referenced tests for content areas not covered by 
the National Teacher Examinations (NTE) published by Educational Testing 
Service. South Carolina also uses the NTE area exams that correspond to 
the certification fields in which they certify teachers. Florida has 
developed one professional knowledge criterion-referenced test which all 
teachers in all fields must pass in order to be certified. Arizona deve- 
loped one basic skills test that is required for certification in all 
fields. Additionally, some states require candidates to pass some compo- 
nents) of the NTE, and many other states are exploring options regarding 
teacher certification and competency testing (Priestley, 1982). 
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One state whose competency certification program has been charac- 
terized as extensive and sophisticated is Georgia (Stoltz, 1981). Georgia 
not only assesses the generic teaching competencies that state deems 
necessary for all teachers seeking certification but also administers 28 
separate criterion-referenced tests, each for testing a different content/ 
specialty area. Among the most recently developed of these tests is the 
Reading Specialist Test. This is a test that Georgia will require of all 
persons at the master's level and beyond who wish to become newly certified 
in reading in the £tate. Since Georgia has been one of the leaders in 
teacher competency testing and has often served as a model for other 
states planning such programs (e.g., Alabama, Oklahoma, South Carolina, 
and West Virginia), we thought it important to look carefully at the 
content objectives of this Georgia test and see how closely its objectives 
correspond with the content emphasized in graduate level reading courses 
in Georgia colleges and universities. 

Test Development 

In 1981 the Georgia Department of Education contracted with National 
Evaluation Systems to develop a criterion-referenced test for certification 
as a reading specialist. The development of the Reading Specialist Test 
involved three major tasks. Ascertaining the domain of knowledge required 
of reading specialists, analyzing their job activities, and constructing 
an appropriate test. 

Defining the domain of knowledge required of reading specialists 
entailed developing a topical outline of that knowledge, elaborating that 
outline, and ultimately formulating 141 measurable objectives. This was 
accomplished by an ad hoc committee of Georgia reading professionals. 



ERIC 



4 



Teacher Competency Testing - 3 

Job analysis involved the rating of all proposed objectives to deter- 
mine the most critical job-related objectives. Two hundred educators 
certified in Georgia as reading specialists were sent surveys. These 
reading specialists were asked to indicate for each objective whether 
they had "taught or used" the content of the objective "during this year 
or the past school year." For those objectives they had taught or used, 
they were asked to rate the amount of time spent teaching or using the 
objective and the extent to which they considered the objective essential 
to their field. Job-relatedness was treated as a two-dimensional construct 
consisting of "time spent" and "essentiality." One hundred and fifty- 
seven, or 79%, of the surveys were returned. Of those returned, 140, or 
70%, were valid. (Only surveys returned by educators who were both 
certified and practicing were considered valid.) One hundred eighteen 
of the 141 proposed objectives were considered to be job-related by the 
reading specialists surveyed. Those 118 objectives were grouped into six 
subareas: (1) Language Development and Reading Readiness, (2) Reading 
Skills, (3) Developmental Reading Instruction, (4) Assessment, (5) Reading 
Disabilities, and (6) Program Management. 

Test construction involved developing test items to measure each of 
the selected objectives. From these objectives National Evaluation Systems 
generated a pool of questions, and then an ad hoc committee of reading 
professionals approved or rewrote these test items. The items were then 
field tested with a sample of 23 students drawn from graduate classes in 
reading at colleges and universities in Georgia. A panel of Georgia 
reading professors and reading specialists then independently reviewed 
each item for content validity, determining whether the question elicited 
a response reflecting knowledge indicated by the corresponding objective. 
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This same panel also participated in standard setting by reviewing the 
field test data and establishing a passing score for the test. Based on 
the field test results and the content validation, National Evaluation 
Systems selected a set of items for the item pool of the Reading Specialist 
Test. 

Rationale for Study 

Throughout the development of the Georgia Reading Specialist Test 
great care was taken to assure that this certification test would not only 
reflect the content knowledge required of reading educators practicing in 
Georgia public schools, but conform to applicable U. S. constitutional 
requirements as well. The consulting firm, National Evaluation Systems, 
recommended the job analysis procedure used for this test in accordance 
with Supreme Court decisions regarding licensure tests and the job- 
relatedness of such tests, as cited in Rubinstein, McDonough, and Allan 
(1982), and the 1978 Uniform Guidelines content (EEOC, CSC, Department 
of Labor, & Department of Justice, 1978). 

The validity and job-relatedness of the test having been established 
by the procedures thus described, our attention now turns to the formal 
training of prospective reading specialists. Does the test reflect the 
substance of their training? In the study reported here our attention 
focused specifically on the relationship between the objectives of the 
Reading Specialist Test and the emphasis placed on these objectives by 
faculty teaching graduate level reading courses in Georgia colleges and 
universities. Although the Georgia Reading Specialist Test was not de- 
signed to be a summative evaluation of one's college preparation, if we 
are to test prospective reading specialists for certification by an 
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instrument constructed from certain content objectives, we are obliged to 
determine the extent to which professors perceive those content objectives 
as important and the extent to which those objectives are taught in graduate 
reading courses. If there is a discrepancy between the content of graduate 
reading courses and the objectives of the job-related certification test, 
then adjustments may well be indicated. 

Description of Study 

A study was undertaken in the Fall, 1982, to examine the correspondence 
between the content objectives of the Georgia Reading Specialist Test and 
the content emphasized in graduate level reading education courses in 
Georgia colleges and universities. A questionnaire was sent to all faculty 
teaching at least one graduate level reading course in Georgia colleges 
and universities with a state-approved program in reading at the master's 
level and/or higher. The names of these faculty were gleaned from a 
directory of graduate faculty in reading education (Blomenberg, 1981) and 
by personal inquiries. In all, 55 questionnaires were sent. 

The 118 job-related content objectives in the six subareas (Language 
Development and Reading Readiness, Reading Skills, Developmental Reading 
Instruction, Assessment, Reading Disabilities, Program Management) of the 
Georgia Reading Specialists Test were collapsed and reduced to 56 ques- 
tionnaire items in an attempt to render the length of the survey manageable. 
Taking a Likert scale format, the questionnaire directed professors to 
indicate the extent to which they address each facet of instruction in 
their graduate reading education classes. There were four options for 
each set of collapsed objectives: no attention, mentioned, stressed, and 
major emphasis. Table 1 summarizes the responses to the questionnaire. 
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Resul ts 

Forty of the 55 questionnaires sent were completed and returned. 
This represents a 69% return. However, of the 40 questionnaires returned, 
only 33 were considered usable for the purposes of this study. Response 
to the questionnaires was assumed to be based upon teaching classes across 
the general areas of language development, developmental reading instruc- 
tion, and diagnosis and remediation of reading problems. In order that 
the results not be misleading, responses were eliminated from the sample 
if they were based upon teaching only narrowly specialized courses. The 
responses summarized in Table 1 and highlighted here by subarea are based 
on 33 usable questionnaires, representing 60% of the total population 
sampled. 

Language Development and Reading Readiness 

In the area of language development and reading readiness, stress or 
major emphasis is given to most instructional areas. Respondents indicated 
that they teach linguistic and concept awareness and that they stress 
auditory, visual, and conmuni cation skills. Additionally, they indicated 
that importance is placed on emotional development and sociocultural and 
motivational factors. About half of the respondents either do not deal 
with specific linguistic, medical, and maturational influences on language 
development or, if they do, they only mention them in their teaching. 

Reading Skills (of Candidates) 

By far the area receiving the least emphasis in all institutions is 
that of developing reading specialist candidates' own reading competence. 
Respondents indicated that they do emphasize becoming aware of one's own 
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use of phonetics, and structural analysis and that they attempt to extend 
comprehension skills and knowledge of appropriate reference sources. 
However, more than half indicated that little if any attention is given 
to students' abilities in spelling, mechanics of writing, graphic skills, 
and following directions. 

Developmental Reading Instruction 

Of all the areas of reading specialist preparation, developmental 
reading instruction clearly receives the major emphasis in all institutions. 
Response to the survey indicates that nearly all professors emphasize the 
development of inferential and critical reading abilities and that nearly 
all attempt to demonstrate a variety of approaches to developing reading 
ability. It appears that in general these approaches are presented under 
the rubric of developing comprehension skills traditionally identified by 
reading teachers and publishers of reading instructional materials. These 
skills include identifying main ideas, recalling details, following se- 
quence, and detecting cause-effect relations. To foster these skills, they 
indicate that they stress the need to accommodate variations in pupils' 
language background, to teach word identification skills, and to inculcate 
study skills. About half the respondents either do not deal with the 
functions of punctuation and capitalization or, if they do, they only men- 
tion them incidentally. 

Assessment 

In the area of assessment, reading specialist preparation appears to 
be strong in Georgia. Most professors indicated that they stress the 
characteristics and uses of different kinds of reading tests, nearly all 
giving special or major attention to informal reading assessment techniques. 
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In general, professors seem not as concerned with technicalities of formal 
assessment as they are with appropriate selection and interpretation of 
these tests. 

Reading Disabilities 

Responses related to teaching about reading disabilities were somewhat 
mixed, as would reflect the controversial nature of this area. In general, 
where particular orientation to dealing with reading problems was left 
open, response was more clearcut. Specifically, two areas strongly em- 
phasized are identifying techniques for determining students' reading per- 
formance and identifying types and functions of materials or equipment used 
in remedial reading. However, where orientation to instruction takes a 
particular bias responses were substantially mixed. For example, matters 
related to either mainstreaming or clinical approaches are at best mentioned 
by a third of the respondents. This is not surprising given the variety 
of theoretical orientations which are represented in Georgia's colleges 
and universities. Theoretically strict orientations are less likely to 
give special attention to propositions considered inconsistent or unimpor- 
tant. One aspect of reading specialist preparation not identified with 
any particular orientation was shown to receive little or no attention by 
over half of the respondents — teaching about purposes and procedures for 
communicating with related specialists regarding students 

« in remedial reading programs. 

Program Management 

Response to the questionnaire indicates that Georgia colleges and 
universities are providing instruction in reading program management, but 
for the most part attention is limited to management concerns related to 
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classroom reading instruction. Most respondents indicated that they deal 
with reading program goals and objectives and identify activities and 
materials appropriate for developmental and remedial reading. Nearly all 
demonstrate readability prediction techniques. Most emphasize methods of 
evaluating and modifying instruction. However, few respondents indicated 
that they give important consideration to promoting interaction among 
teaching, supervisory and administrative personnel. Only a handful of 
respondents said that they deal with Public Law 94-142, Education of the 
Handicapped Act. 

Conclusions 

It seems reasonable to expect that reading specialist candidates will 
be successful on test areas/objectives in which stress or major emphasis 
was placed in their coursework. Conversely, on areas/objectives in which 
faculty indicated no emphasis, or as only mentioning in their graduate 
courses, potential deficiencies might exist. For instance, the cumulative 
responses of 60% of the graduate reading faculty in Georgia indicates 
that about half or less than half of them put little emphasis on: specific 
linguistic, medical, and maturational influences on language development; 
the reading specialist candidates* own reading and study skills abilities; 
functions of punctuation and capitalization; concepts of test validity and 
reliability; cooperation with other professional personnel and knowledge 
of relevant professional affiliations/ publications; and Education of the 
Handicapped Act (Public Law 94-142). 

A coimon thread of weakness across the subareas was in objectives 
dealing with cooperation with other professionals. Additionally, there 
is evidence that little attention is paid to teaching reading specialists 
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about other professional fields which are related to reading. The Handi- 
capped Act is only mentioned, or not mentioned at all, in graduate reading 
coursework. 

Generally, the results suggest that the reading specialist candidates 
in Georgia should do well on most of the objectives of the Reading Special- 
ist certification test. Subarea responses looked very positive for all 
subareas, with the possible exception of "Reading Skills." If these 
faculty perceptions are accurate, and if institutions have not in some way 
assured students' competence in the reading/study skills, statewide test 
results are likely to indicate weaknesses in candidates' own reading 
abilities and study skills. 
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Table 1 

Emphasis Reading Professors Place on 
Areas of Preparation for Reading Specialist 
Certification in Georgia 3 



Area of ^ 
Preparation 

Language Development and 
Reading Readiness 

Phonology, morphology, 
semantics, and syntax 

Psychomotor skills 

Linguistic awareness 

Auditory and visual 

di scrinri na ti on/percepti on 

Auditory and visual memory 

Oral conmuni cation skills, 
concept awareness, and 
reading readiness 

Sociocultural factors 

Emotional development 

Interest/motivation 

Health and physical 
development 

Reading Skills of Candidates 
Phonics 

Structural analysis 

Spelling irregularities 

Synonyms, antonyms, homonyms 

Word meaning using context 

Word meaning using 
dictionary 

Comprehension skills 



No Major 
Attention Mentioned Stressed Emphasis 
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12 


10 


*• 
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4 


13 


15 
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16 


11 
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13 


12 
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23 
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11 


18 


1 


8 


18 
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3 
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16 
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1 


2 


16 


14 


2 


14 


13 


4 


2 


5 


14 


12 


1 


6 


14 


12 


3 


13 


11 


6 


1 


6 


15 


11 


1 


2 


14 


16 


4 


13 


11 


5 


1 


2 


10 


20 
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Area of h No 
Preparation 0 Attention 


Mentioned 




Major 


Reading Skills of Candidates (continued) 








Punctuation/capi tal izati on 
and effect on passage 


2 


13 


16 


2 


Following directions 


3 


13 


10 


7 


Graphic interpretation 


2 


14 


14 


3 


References sources 


i 
i 


in 
1U 


ID 


c 
o 


Developmental Reading Instruction 










Variety of approaches to 
teaching reading 


n 

u 


C 


7 




Techniques to accommodate 
different language patterns 


3 


A 
■f 


19 


7 
# 


Techniques for word 
identification 


0 


1 
1 


16 


16 


Techniques for developing 
vocabulary 


1 


10 


14 


8 


Techniques for developing 
literal comprehension 


1 


2 


10 


20 


Techniques for developing 
interpretation of punctuation/ 
caoi tal i zati on 


5 


U 




3 


Techniques for developing 
inferential comprehension 


0 


0 
c 


12 


19 


Techniques for developing 
critical reading 


A 

u 


2 


14 


17 


Techniques for developing 
literary appreciation 


1 


3 


17 


12 


Techniques for developing 
reading/listening skills 


0 


3 






Assessment 










Norn- referenced and criterion- 
referenced uses of tests 


1 


6 


17 


9 


Attitude/interest tests 


1 


4 


16 


12 


Observation, cloze, and IRI 


1 


1 


10 


21 


Concepts of validity and 
reliability 


3 


15 


14 


1 


Selection of tests 


2 


2 


17 


12 
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Area of ^ 
Preparation 


No 

Attention 


Mentioned 


Stressed 


Major 
Emphasis 


Reading Disabilities 










Physical, psychological, 
intellectual, or socio- 
economic factors 


2 


6 


12 


13 


Demonstrate an understanding 
of observation or referral 
procedures, and uses of 
visual /auditory screenings 


3 


7 


13 


10 


Independent, instructional, 
frustration, ana potential 
levels 


1 


1 


10 


21 


Selection, scneaunng, ana 
grouping remedial instruction 


3 


7 


17 


6 


Motivational btra Ley ica 


o 


8 


19 


6 


Integrating remedial reading 

Wl tni n regular uiaooruuiu 


o 


10 


16 


7 


Monitoring/ recording student 
progress in remedial reading 


4 


10 


16 


3 


Communicating with other 
professionals and agencies 


Q 
O 


n 


1C 


2 


Remedial reading materials/ 
equipment 


1 


6 


19 


7 


Program Management 










Remedial, developmental 
enrichment programs 


0 


4 


23 


6 


Interaction with students, 
parents, classroom leatncri, 
administrators/support staff 


7 


12 


12 


2 


Goals of comprehensive 
reading curriculum 


2 


8 


15 


8 


Evaluating reading programs 


5 


4 


21 


3 


Scheduling reading program 


4 


13 


12 


4 


Placement of students 


3 


7 


19 


4 


Education of the Handicapped 
Act (Public Law 94-142) 


10 


15 


8 


0 


Selection of materials 


1 


8 


17 


7 


Readability level 


0 


6 


23 


4 


Selection of literature 


2 


9 


13 


9 


Professional organizations 
and publications 


2 


15 


12 


4 



